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[57] ABSTRACT 

The invention provides a display apparatus which operates 
in a synchronized relationship with a movement of the body 
of a viewer. A movement amount or a turning angle of the 
entire body or a part of the body of a viewer such as, for 
example, a movement of the eye-balls or a turning angle of 
the head, is detected by a sensor, and a portion of an original 
image is extracted and displayed in response to a magnitude 
of the detected movement amount or turning angle. 
Simultaneously, an audio signal is processed based on the 
information from the sensor so that an audio image may be 
formed at a normal position corresponding to the displayed 
image. 

9 Claims, 8 Drawing Sheets 
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DISPLAY APPARATUS OPERABLE IN It is a second problem of the conventional image process- 

SYNCHRONISM WITH A MOVEMENT OF ing apparatus that a displacement is produced between 

THE BODY OF A VIEWER normal positions of a video image and an audio image. In 

particular, where a fixed type image reproduction apparatus 

BACKGROUND OF THE INVENTION 5 wherein an imag6 is not v ^ ied by b a t ^ reamt of head 

This invention relates to a display apparatus such as, for °f a person like a television receiver or the like having an 

example, a head mounted display (HMD) apparatus or a ordinary CRT display tube is used in combination with a 

television receiver which reproduces and displays a video headphone wherein the normal position of an audio image is 

signal in the form of a digital signal transmitted and received varied by a movement of the head of a person, since only the 

as a compressed signal used, for example, in satellite broad- 10 normal position of the audio image is varied in response to 

casting and/or outputs sound,, and more particularly to a a movement of the head, depending upon the accuracy or the 

display apparatus which calculates a turning angle, an like of a gyro sensor which detects the turning angle of the 

amount of movement of the head of a viewer in the direction head, a displacement is possibly produced between the 

of the visual point and so forth, extracts and reproduces an position of a video image and the normal position of the 

image supplied with the aspect ratio of 16:9 or 4:3 on the real 15 auc ho image. 

time taking an amount of movement of the image and Also where a head mounted display unit wherein a video 

processes audio information in response to the amount of image is not varied by a movement of the head is used in 

movement to provide the feeling of presence to the viewer combination with a headphone of the type described above, 

at a comparatively low cost. when the head is moved, the video image is not varied, and 

In the field of the image technology of AV (audio-visual) 20 consequently, the position of the video image is displaced 

apparatus or AV systems, an image reproduction method is fr 0D1 tne normal position of the audio image, 

conventionally known wherein an image being observed by It is to be noted that the normal position of an audio image 

a viewer is varied in response to a turning movement of the signifies the position of a virtual sound source formed in the 

head or some other movement of the viewer. head of a viewer when the viewer listens to an audio signal 

Meanwhile, the MPEG2 format of a data compression 25 originally adjusted so as to be reproduced from an audio 

system for processing an image of a moving object have outputting apparatus such as a loudspeaker or the like using 

been standardized in the world, and data encoders and data a headphone, such as, for example, a front right position, a 

decoders conforming to the standards have been marketed front left position or a front central position, 

already. An MPEG decoder has a PAN & SCAN function. _ cttx^ahv ^ ^tttt TXTWr . xr ™ NT 

™- t *• ** a • *• ■ r 30 summary of the invention 

I ins function writes region designating information repre- 
sentative of an aspect ratio of a display screen in a header It is an object of the present invention to provide a display 
part of a video signal defined on the MPEG2 decoder to apparatus operable in synchronism with a movement of the 
designate an aspect ratio of a display image. body of a viewer which solves the problems described above 
Making use of this function, it is possible to extract image 35 and eliminates a possible displacement between the normal 
information of the aspect ratio of 4:3 from compressed positions of a video image and an audio image with a 
image information of the aspect ratio of 16:9 of the MPEG2 comparatively low cost. 

format. In order to attain the object described above, according to 

On the other hand, in the field of the audio technology of the present invention, there is provided a display apparatus 

AV apparatus, a method is known wherein, as disclosed in 40 operable in a synchronized relationship with a movement of 

Japanese Patent Laid-Open Application No. Heisei 7-95698 a bod y of a viewer, comprising display means for displaying 

entitled "Audio Reproduction Apparatus" assigned to the a portion of an original image on a screen, means for 

same assignee of the present invention, when sound col- detecting a movement amount or a turning angle of at least 

lected in stereo is to be reproduced by means of a a part of a body of a viewer of the display means, and video 

headphone, the left and right acoustic outputs of the head- 45 processing means for extracting a portion of the original 

phone are corrected with impulse responses stored in a image in response to a magnitude of the detected movement 

memory in advance in response a turning angle of the head amount or turning angle and supplying the extracted portion 

of a listener. °f me original image to the display means. 

Further, as a sensor for detecting a movement amount or The display apparatus may further comprise audio repro- 

a turning angle of the body of a person, various sensors are 50 Auction means, and audio processing means connected to the 

known including such a digital angle detector which makes audio reproduction means for processing an audio signal in 

use of a horizontal component of force of the terrestrial response to the magnitude of the detected movement amount 

magnetism as disclosed in Japanese Patent Laid-Open or turning angle so that sound may be outputted from a 

Application No. Heisei 7-95698 mentioned hereinabove, position corresponding to the image extracted by the video 

and a sensor which employs a gyro. 55 processing means. 

However, according to the conventional image processing The display apparatus may be constructed such that the 

apparatus described above, in principle, a turning angle of original image is obtained by reproduction of a compressed 

the head of a person is detected by means of a sensor, and digital video signal, and the image processing means 

a special video signal which corresponds to a number of decodes a portion of the digital video signal to extract the 

images several to several tents times a display screen of an 60 portion of the original image. 

AV apparatus is processed in synchronism with the speed of In the display apparatus, since a portion of an original 

the turning movement based on the turning angle informa- image to be displayed is extracted from the original image 

tion from the sensor. Consequently, a very large memory in response to a movement of a part of the body of a viewer 

capacity and a high processing speed are demanded, and this such as the eye-balls or a movement of the entire body and 

makes an obstacle to minimization and reduction in weight 65 the extracted image is displayed, the screen can be varied in 

and in cost of an AV apparatus, which is a first problem of response to a movement of the body such as, for example, 

the conventional image processing method. the eye -balls. Consequently, the display apparatus is advan- 
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tageous in that it can display an image which provides a 
promoted feeling of presence. 

Further, since also the sound is varied in response to the 
varied screen so that the normal positions of the image 
observed by the viewer and an audio image formed in the 5 
head of the viewer may conform with each other, the display 
apparatus is advantageous also in that it can construct, for 
example, a virtual real space which includes sound and an 
image which conform with each other and provides a good 
feeling of presence to the viewer of the display apparatus. 10 

Further, since the display apparatus can be constructed 
without requiring any complicated structure, an otherwise 
possible displacement between video and audio images can 
be eliminated in an AV apparatus as the display apparatus 
which includes an existing headphone and an existing 15 
MPEG2 decoder chip by which, for example, the position of 
an audio image can be varied. 

The above and other objects, features and advantages of 
the present invention will become apparent from the fol- 
lowing description and the appended claims, taken in con- 
junction with the accompanying drawings in which like 
parts or elements are denoted by like reference characters. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 

FIGS. 1A and IB are a schematic top plan view and a 
schematic side elevational view showing an AV system to 
which the present invention is applied; 

FIG. 2 is a schematic perspective view showing an 
another AV system to which the preseat invention is applied; 30 

FIG. 3 is a schematic elevational view showing a further 
AV apparatus to which the present invention is applied; 

FIGS. 4A, 4B and 4C are schematic illustrations showing 
different examples of a blanking display; 

FIGS. 5A, 5B and 5C are schematic illustrations showing 35 
different examples of an extraction display; 

FIGS. 6 A, 6B and 6C are schematic illustrations illus- 
trating a relationship between a turning movement of the 
head of a viewer and a blanking display; 

FIG. 7 is a block diagram showing a basic construction of 40 
an AV system to which the present invention is applied; 

FIG. 8 is a block diagram showing an ordinary digital 
video signal processing apparatus; 

FIG. 9 is a flow diagram showing a processing apparatus 45 
for a video signal of the MPEG2 format; and 

FIG. 10 is a block diagram showing a basic construction 
of an audio processing apparatus incorporated commonly in 
the AV apparatus of FIGS. 1A and IB to 3. 

DESCRIPTION OF THE PREFERRED 50 
EMBODIMENTS 

Appearances of different AV systems as display apparatus 
to which the present invention is applied are shown in FIGS. 
1A and IB, 2 and 3. 55 

Referring first to FIGS. 1A and IB, the video display 
apparatus shown is an AV system 100 which includes a 
display unit of the head fixed type, that is, a head mounted 
display (HMD) unit. The AV system 100 has a head set 
structure by which it is fixed to the head of a viewer 2 by 60 
means of some support structure 1 such as a head band, and 
includes a display unit 3 provided in front of the two eyes of 
the viewer 2 for displaying a predetermined image on a 
screen, a stereo headphone 4 including two sounding ele- 
ments for contacting with the left and right ears of the viewer 65 
2, and a gyro sensor 5 for detecting a movement of the head 
of the viewer 2. 



4 

An image displayed on the display unit 3 is an image in 
a certain partial region of a "horizontally elongated full 
image region" (hereinafter referred to as original image), 
and if the viewer 2 is turned, for example, in the rightward 
direction, the region of the image displayed on the display 
unit 3 in the original image moves in the rightward direction 
by an amount of movement corresponding to the turning 
movement of the head. 

Generally, two different methods are available to extract 
an image to be displayed from an original image. 

(1) According to the first technique for extracting an 
image to be displayed from an original image, a blanking 
portion 9 or 10 is provided at a left or right end portion of 
the original image as shown in FIG. 4A, 4B or 4C so that an 
image of a desired region is displayed consequently. 

In particular, if the amount of variation of the turning 
angle of the head exhibits a movement in the leftward 
direction, then the blanking portion 10 on the right side is set 
with a comparatively large width while the blanking portion 
9 on the left side is set with a comparatively small width or 
with no width as seen in FIG. 4A 

If the amount of variation of the turning angle of the head 
is zero and the head does not move leftwardly or 
rightwardly, then the blanking portions 9 and 10 at the left 
and right end portions are set equal to each other as seen in 
FIG. 4B. 

On the other hand, if the amount of variation of the 
turning angle of the head exhibits a movement in the 
rightward direction, then the blanking portion 9 on the left 
side is set with a comparatively large width while the 
blanking portion 10 on the right side is set with a compara- 
tively small width or with no width as seen in FIG. 4C. 

In this manner, a construction wherein the widths of the 
blanking portions 9 and 10 on the left and right ends of the 
screen are varied in response to an amount of movement of 
the turning angle of the head so that it looks as if the screen 
displayed were moved in response to the amount of move- 
ment of the turning angle of the head can be realized. 

(2) According to the second technique for extracting an 
image to be displayed from an original image, an image of 
a desired region is extracted from the original image and 
displayed in such a manner as shown in FIGS. 5 A, 5B and 
5C. This technique is based on the PAN & SCAN function 
which is a standard of a format function of a compressed 
transmission/reception signal according to the MPEG2 for- 
mat employed in satellite broadcasting in Japan, which will 
be hereinafter described. According to the MPEG2 format, 
a digital video signal of the aspect ratio of 16:9 is divided 
into 256 segments and includes a control signal indicative of 
the start and the end of each of the divided video signals, and 
an image is extracted and displayed in accordance with the 
control signal in response to an amount of a turning angle of 
the head of a viewer. 

In particular, if the turning angle of the head exhibits an 
amount of movement in the leftward direction, then the 
control signal indicating a start and an end is used to extract 
a control signal of a left side portion is extracted as seen in 
FIG. 5Aso that an image portion lOAon the right side may 
not be displayed. As a result, an image of a left side portion 
of the original image is displayed on the screen. 

On the other hand, if no amount of movement of the 
turning angle of the head is detected, then an image which 
does not include left and right image portions 9A and 10A 
is displayed as seen in FIG. 5B. Thus, an image at the center 
of the original image is extracted and displayed. 

Or, if the turning angle of the head exhibits an amount of 
movement in the rightward direction, then an image which 
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does not include an image portion 9A on the left side of the The AV system 300 includes a stereo headphone 4 and a gyro 

original image is displayed. Consequently, a right side sensor 5 attached to the head of a viewer 2 similarly as in the 

portion of the original image is extracted and displayed. display apparatus of FIGS. 1A and IB, and an ordinary 

In this manner, similarly to the first technique wherein the television receiver 8 as a floor type display unit which 

widths of the blanking portions 9 and 10 shown in FIGS. 4A 5 eludes a display section for displaying an image on a 

A M ^ . , . • * t , j > screen and a sound outputtmg section for outputting sound, 

to 4C are varied, if an image is extracted and displayed to « j ♦ i ♦ u* u 

AU ,. , u*u„* ■ — „* iu„ \„a «„t,< The gyro sensor 5 corresponds to a sensor element which 

vary the screen display such that images at the left and right on ^ of viewers t forwardl 

ends of the ongmal image are cut away in response to an dis of ^ television g and 

amount of movement of the turning angle of the head, then a variation amount of a turni le of a vicwi direction, 

the screen display can be varied in response to the turning w ^ ^ s may be formed in1c ^, y with the stereo 

movement of the head of the viewer. headphone 4 or may otherwise be attached to a portion of the 

Referring now to FIG. 2, there is shown another display body of the viewer only if it can detect the viewing direction 

apparatus to which the present invention is applied. The of the viewer. 

display apparatus is an AV system 200 which includes a ] n this instance, an image of the aspect ratio of, for 

display unit integrated with a viewer. The AV system 200 example, 16:9 which corresponds to the full screen of the 

includes a stereo headphone 4 attached to the head of a television receiver 8 is used as an original image, and by 

viewer 2 similarly as in the display apparatus of FIGS. 1A providing the blanking portions 9 and 10 of variable widths 

and IB, and a display unit 6 of the screen type. The display on the opposite left and right sides of the original image 

unit 6 is secured to an end portion of a structure 7. The ( re f er to FIGS. 4Ato 4C) or by extracting a screen from the 

viewer 2 can be seated on a viewing seat in the form of a sofa original image (refer to FIGS. 5A to 5C), an image of a 

secured to the other end portion of the structure 7 while partial area of the original image, for example, an image of 

facing the screen of the display unit 6. The viewing seat on the aspect ratio of 4:3, is extracted and displayed, 

the structure 7 is turned integrally with the display unit 6 In this manner> by varying the widths of the left and right 

serving as a display section. ^ blanking portions 9 and 10 in response to an amount of 

The structure 7 can be turned in the opposite leftward and movement of a turning movement of the head of the viewer 

rightward directions as indicated by a double-sided arrow 2 shown in FIGS. 1A and IB or FIG. 3 or in response to an 

mark around a center axis la and detects a turning angle in amount of movement when the screen display and the 

the leftward or rightward direction. Accordingly, when the viewer are moved together with each other in the leftward or 

structure 7 is turned, the display unit 6 and the viewer 2 3Q rightwardly direction without varying the viewing direction 

move simultaneously with each other while the viewer 2 0 f the viewer shown in FIG. 2, an image region to be 

observes an image on the display unit 6 without turning the extracted from an original image is varied. For example, if 

head or the like thereof. If the structure 7 is turned, for the viewer 2 turns the head in the rightward direction as seen 

example, in the rightward direction, then an image region to m FIG. 6A, the width of the blanking portion 9 on the left 

be displayed on the display unit 6 is moved in the rightward 35 side is set comparatively wide while the width of the 

direction in the original image by an amount corresponding blanking portion 10 on the right side is set comparatively 

to the amount of movement caused by the turning movement narrow or reduced to zero. If the head of the viewer 2 is 

of the entire body. Where the body of the viewer positioned directed just forwardly as seen in FIG. 6B, the widths of the 

forwardly of the screen is turned integrally in the leftward or left and right blanking portions 9 and 10 are set equal to each 

rightward direction and an image is extracted from the other. If the viewer 2 turns the head leftwardly as seen in 

original image as seen from FIGS. 4A to 4c or 5 A to 5 C to FIG. 6C, then the width of the blanking portion 9 on the left 

vary the screen display in such a manner as described above, side is set comparatively narrow or reduced to zero while the 

even when the entire body of the viewer is physically moved width of the blankiog portion 10 on the right side is 

in the leftward or rightward direction without turning the increased. Consequently, an image being displayed looks as 

head, a screen which provides a feeling of presence to the ^ if it were moving. 

viewer can be displayed. It is to be noted that, if audio ] n 0 f tne AV system 100 (refer to FIGS. 1A and IB), 

processing is performed so as to establish synchronism with th e AV system 200 (refer to FIG. 2) and the AV system 300 

the varying screen display, a normal position of an audio ( rc f er to piQ. 3), a movement of the entire body or a portion 

image conforming to the varying screen display can be of the body of the viewer 2 is detected by means of a sensor, 

obtained. 50 an d an output of the sensor is inputted to a microcomputer, 

Here, the normal position of the audio image signifies, as by which body movement information such as the direction, 
described in the description of the background of the inven- speed and amount (turning angle) of movement or turning 
tion hereinabove, the position of a virtual sound source movement and so forth is calculated. It is to be noted that the 
formed in the head of a viewer when the viewer listens to an microcomputer may be provided at a suitable location on the 
audio signal originally adjusted so as to be reproduced from S5 display unit 3, 6 or 8 side or on the stereo headphone 4 side, 
an audio outputting apparatus such as a loudspeaker or the The body movement information calculated by the micro- 
like using a headphone, such as, for example, a front right computer is supplied commonly to the stereo headphone 4, 
position, a front left position or a front central position. which outputs sound to which audio processing has been 

It is to be noted that, while, in the display apparatus of performed, and the display unit 3, 6 or 8 which performs 

FIG. 2, the turning angle of the center axis la of the structure 60 image processing. 

7 is detected, for example, the stereo headphone 4 may Each of the display unit 3, 6 and 8 calculates, based on the 

include an integral gyro sensor in order to detect a turning body movement information, an image region to be 

angle of a viewer, extracted from an original image, and outputs a region 

Referring now to FIG. 3, there is shown a further display designation signal which designates the calculated image 

apparatus to which the present invention is applied. The 65 region. The image region designated by the region design a- 

display apparatus is anAV system 300 which includes a floor tion signal is extracted from the original image and dis- 

type display unit such as, for example, a television receiver. played on the display screen. 
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Simultaneously, the stereo headphone 4 forms a normal be outputted from the storage apparatus 14. The storage/ 
position of an audio image based on the body movement control apparatus 12 simultaneously sends a write control 
information and processes an audio signal so that an audio signal to the storage apparatus 15 to write video information 
image which conforms to the image displayed on the display then into the storage apparatus 15. This sequence of opera- 
unit 3, 6 or 8 may be formed. 5 tions is repeated for each synchronizing signal. 

Subsequently, a construction of a display apparatus for li is to be Qoted ^ while ^ ^P 1 *? apparatus shown 

performing the image processing described above will be in nG - 7. is constructed such that a video signal to be 

described in detail displayed is extracted from video information stored in the 

„ . . ^ T " m , . , . . . e storage apparatus 14 or 15 designating a region for such 

Referring to FIG. 7 there * shown a basic construction of cxttaction ^ taatively a ^ signal extracted from 

a display apparatus for performing the image processing ^ yideo designatirig a region for such ej!t rac- 

according to the present invention. The display apparatus ^ ^ ^ stored ^ ^ e ^ M of 

shown includes a synchronmng signal inputting section 11, ^ ^ ^ ^ desa3oed in Processing of an ordinary 

a storage/control apparatus 12, a video information inputting digital video signal" given below. 

section 13, a pair of storage apparatus 14 and 15, and an , . £ „ . . . c <. . t . . , 

e rocessin section 18 15 1 n the folio wing, transmission- reception ol a digital signal 

image processing ec 10 . which is employed in satellite broadcasting and so forth by 

The synchronizing signal inputting section 11 extracts a ^ dispky apparatus having the basic construction as 

synchronizing signal from a digital video signal of an dcscribed aboV e will be described in 1. Processing of an 

original image and outputs the synchronizing signal to the or dinary digital video signal and 2. Processing of a video 

storage/control apparatus 12. M signa] of thc MpEG2 fonnat 

The storage/control apparatus 12 controls so that, each i. Processing of an Ordinary Digital Video Signal 
time a synchronizing signal is inputted thereto, image infor- ^ image processing apparatus which performs a method 
mation may be written into one of the storage apparatus 14 0 f processing an ordinary digital video signal is shown in 
and 15. Further, the storage/control apparatus 12 receives, FIG. 8. Referring to FIG. 8, the image processing apparatus 
simultaneously with the synchronizing signal, a region des- 25 includes a storage/control apparatus 12 and a pair of storage 
ignation signal for controlling a region of the storage appa- apparatus 14 and 15 described hereinabove, a video signal 
ratus 14 or 15 into which video information is to be written conversion apparatus 16 connected commonly to outputs of 
based on region information from the image processing the storage apparatus 14 and 15, a synchronizing signal 
section 18 which calculates a turning movement amount of detection apparatus 17, a control microcomputer 18 serving 
a turning angle and so forth. 30 as an image processing section which performs image 
The video information inputting section 13 receives processing. It is to be noted that the storage/control appa- 
image information of a digital video signal of an original ratus 12 may be included in the control microcomputer 18 as 
image and outputs the image information to the storage indicated by broken lines in FIG. 8, 
apparatus 14 and 15. The control microcomputer 18 service as an image pro- 
Each of the storage apparatus 14 and 15 writes, when a 35 cessing section includes a synchronizing calculation section 
write control signal is received from the storage/control ™ outputting a synchronizing signal, an angle calcula- 
apparatus 12, all of image information from the video tion section 20, an extraction start-end timing calculation 
information inputting section 13 or part of the image infor- section 21, and an angle variation amount adjustment sec- 
mation designated by region information from the storage/ tion 22 for adjusting an angle variation amount based on a 
control apparatus 12. It is to be noted that whether all or part 40 screen size and an image-visual point distance. The control 
of the image information is to be written depends upon a microcomputer 18 which includes the components just men- 
difference in construction, and this will be hereinafter tioned has a very high operation speed comparing with the 
described period of a synchronizing signal of a video signal so that all 

„ . 1C , 4 , #t - mn „ a required calculations can be completed within a time 

The image processing section 18 detects a turning move- , M , ,. t 1. • • • 1 

* * . , r.Lujf • ■) 45 between two adjacent synchronizing signals, 

ment amount of a turning angle of the head of a viewer and J . . , . . ^« > 

r * & T , , . 1 The synchronizing calculation section 19 receives a syn- 

oerforms processing of an image based on the detected , . / . , f , t , , . , . . . - 

F 6 chronizing signal detected by the synchronizing signal 

movement amount. detection apparatus 17 and outputs the received synchroniz- 

The image processing section 18 generates a read control ing sigQa] tQ the storage/control apparatus 12 and the extrac- 

signal for designating whether all of video information 5Q ^ start . end timing calculation section 21. 

stored in the storage apparatus 14 or 15 or part of the video The ca i cu i at ion section 20 receives information of 

information which is in a region designated by a region a posiuon> a direction, a speed, an angle and so forth from 

designation signal from the storage/control apparatus 12 ^ gyro sensof ^ calculales a turning angle of the head of 

should be outputted. It is to be noted that whether all or part the viewer 2 and a visual point movement distance, and 

of the video information is to be outputted depends upon a 55 QUtputs a result of ^ ca i culation) f or example, to the stereo 

difference in construction, and this will be hereinafter headphone 4 (refer to FIGS. 1 A and IB) via the extraction 

described. start-end timing calculation section 21 and the audio pro- 

The display apparatus having the construction described cessing section not shown, 

above with reference to FIG. 7 operates in the following xhe extraction start-end timing calculation section 21 

manner. 60 produces a region designation signal representative of an 

In particular, when a synchronizing signal is inputted, the image region of an original image from which an image is 

storage/control apparatus 12 sends a write control signal to to be extracted based on an angle signal calculated by the 

the storage apparatus 14 to store video information. The angle calculation section 20, that is, representative of tim- 

storage/control apparatus 12 sends a read control signal to ings of the start and the end for extraction, and outputs the 

the storage apparatus 14 in response to a region designation 65 region designation signal to the storage/control apparatus 12 

signal which is generated simultaneously with a next syn- and the storage apparatus 14 and 15. It is to be noted that 

chronizing signal so that the stored video information may writing and reading control into and from the storage 
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apparatus 14 and 15 is omitted here since it is described angle information from the gyro sensor 5 or the like calcu- 

hereinabove with reference to FIG. 7. lated by the angle calculation section 20, and outputs the 

Hie angle variation amount adjustment section 22 calcu- extraction start position as a region designation signal to the 

lates a correction value for an angle variation amount based MPEG2 decoder 23. The extraction start position can be 

on a screen size and a distance to the visual point of the 5 selected arbitrarily from among start and end positions at 

viewer and outputs the calculated correction value to the which the original image is divided into 256 segments, 

extraction start-end timing calculation section 21. The processing apparatus for a video signal of the 

Thus, a video signal in a region designated by a region MPEG2 format having the construction described above 

designation signal outputted from the control microcom- with reference to FIG. 9 operates in the following manner, 

puter 18 serving as an image processing section having the 10 Video information of the MPEG2 format included in 

construction described above is outputted from the storage information inputted as a result of reproduction of a package 

apparatus 14 or 15 to the video signal conversion apparatus medium or reception of a broadcast is inputted to the 

16. In this instance, the following two sequences are MPEG2 decoder 23 while audio information is supplied, for 

available, and either one of the sequences may be employed. example, to the stereo headphone 4 (refer to FIGS. 1 A and 

In particular, 15 IB) which outputs sound as hereinafter described. 

(1) the storage/control apparatus 12 stores video informa- While a header part of the MPEG2 format inputted to the 
tion of an original image as it is into the storage MPEG2 decoder 23 includes region designating information 
apparatus 14 or 15, and the storage apparatus 14 or 15 for cutting out an image of the aspect ratio of 4:3 from image 
outputs the image information in a region designated by information of the aspect ratio of 1 6: 9, contents of the region 
a region designation signal from the control microcom- 20 designating information are not used by the present process- 
puter 18; or ing apparatus, but a region designation signal produced by 

(2) the storage/control apparatus 12 stores video informa- the control microcomputer 25 is written into the place for the 
tion only for an image region designated by region region designating information. 

designation information from the control microcom- A video signal in a region designated by the region 

puter 18 into the storage apparatus 14 or 15, and the 25 designation signal is outputted from the MPEG2 decoder 23. 

storage apparatus 14 or 15 outputs the stored contents The video signal is converted into an analog signal by the 

as they are. digital to analog converter 24 and then sent as a displaying 

In either case, since the image information outputted from video signal to the display section not shown. As a result, the 

the storage apparatus 14 or 15 is reduced in information image in the designated region is displayed on the display 

amount comparing with that of the original image, either the 30 section. 

information amount is compensated for by the video signal Subsequently, the audio processing section will be 

conversion apparatus 16 or information is outputted varying described. 

the reading speed of information from the storage apparatus The audio processing section is a modification to and is 

14 or 15. In this instance, depending upon the recording different from the audio processing section disclosed in 

method, a synchronizing signal is either superposed or 35 Japanese Patent Laid-Open Application No. Heisei 7-95698 

re-produced. mentioned hereinabove in that information of a movement 

Since information outputted from the storage apparatus 14 or a turning movement of a viewer is supplied from the angle 

or 15 is in the form of a digital signal, if necessary, the video calculation section 20 (refer to FIG. 8 or 9) of the video 

signal conversion apparatus 16 is provided with a digital to processing section described hereinabove. Thus, since 

analog conversion function. 40 detailed description is given in the document mentioned 

The display image signal outputted from the video signal above, only an outline of the audio processing section will 

conversion apparatus 16 is sent via an image signal process- be described with reference to FIG. 10 in which a very 

ing circuit not shown to and displayed on the display unit. simplified block diagram is shown. 

2. Processing of a Video Signal of the MPEG2 Format Referring to FIG. 10, the audio processing apparatus is 

Referring to FIG. 9, a processing apparatus for a video 45 constructed, for example, as a stereo headphone 4 and 

signal of the MPEG2 format includes an MPEG2 decoder generally includes an address control section 30, a memory 

23, a digital to analog (D/A) converter 24, and a control 31, a convolution integrator 32, another memory 33, a digital 

microcomputer 25 serving as an image processing section to analog (D/A) converter 34 and a sounding body 35. 

for performing image processing. The address control section 30 receives an angle signal 

The control microcomputer 25 for performing image 50 outputted from the angle calculation section 20 described 

processing receives, similarly to the control microcomputer hereinabove (refer to FIG. 8 or 9) and designates an address 

18 described hereinabove with reference to FIG. 8, infor- of the memory 31 based on the received angle signal. The 

mation of a position, a direction, a speed, an angle and so angle and the address are coordinated in a one-by-one 

forth from the gyro sensor 5, calculates a turning angle of the corresponding relationship. Where a common means is used 

head of a viewer 2 and a visual point movement distance, 55 for detecting a turning angle and so forth to be used for video 

and outputs a result of the calculation, for example, to a processing and audio processing, that is, for detecting varia- 

stereo headphone 4 (refer to FIGS. 1A and IB) via the don amounts in movement and/or turning angle of the entire 

extraction start-end timing calculation section 21 and the body or part of the body of a viewer in such a manner as 

audio processing section not shown. described above, the viewer is prevented from feeling a 

The control microcomputer 25 includes an angle calcu- 60 fluctuation in detection amount which arises from a disper- 

lation section 20, an extraction position calculation adjust- sion of a part such as a gyro sensor or a temperature, 

ment signal generation section 26, and an angle variation A correspondence table between the angle and the 

amount adjustment section 22 for adjusting an angle varia- impulse response is stored in the memory 31. The impulse 

tion amount based on a screen size and an image-visual point response represents a virtual sound source position with 

distance. The extraction position calculadon adjustment 65 respect to a reference direction of the head of a viewer, 

signal generation section 26 calculates an extraction start Accordingly, in response to an angular position of the head, 

position of a video signal from an original image based on an impulse response signal representative of a virtual sound 
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source position conforming to the direction of an image is 
outputted from the memory 31. 

The impulse response signal is supplied to the convolu- 
tion integrator 32 and the memory 33. The convolution 
integrator 32 and memory 33 convolutionally integrate the 
impulse response signal with respect to an audio signal 
inputted thereto to produce an audio signal representative of 
an audio image having a normal position set in response to 
the angle of the head and outputs the audio signal to the 
digital to analog converter 34. 

Hie audio signal is converted into an analog signal by the 
digital to analog converter 34 and supplied to the sounding 
body 35, from which it is supplied as sound to the ears of the 
viewer. 



10 



where an apparatus which includes a headphone which can 
vary an audio image position such as a headphone known 
under the trade name of, for example, ORBIT or Virtual- 
phone and an MPEG2 decoder chip is used, a system can be 
constructed only if software is modified suitably. Also it is 
possible to construct a virtual real space. 

Further, an operation screen may be provided at an end 
portion of a screen of a screen display unit such as, for 
example, a television receiver, such that, by detection of a 
turning angle of the head of a viewer, a screen for setting or 
changing can be observed while the eye-balls are observing 
the operation screen on the screen. 

Further, if a sensor for detecting a visual point position on 
a screen is provided, then an operation screen provided at an 



While description overlaps, an image is extracted from an 15 end portion of the screen for performing setting or changing 
original image and displayed such that, as shown in FIGS. of the screen can be operated simply by a movement of the 
4A to 4C or 5Ato 5C, with reference to a center position P eye-balls. 

with reference to which the original image is to be displayed Further, since screen and audio processing is performed in 
on the screen, a moved position Q with respect to the current a synchronized relationship with a movement of the body of 
screen is calculated, for example, from a movement amount 20 a viewer, a possible bad influence upon the body can be 
of a turning angle of the head, and the original image is reduced. 

displayed cutting away images at the opposite left and right Furthermore, since screen and audio processing is per- 
end portions thereof with reference to the moved position Q. formed in a synchronized relationship with a movement of 
The image portion after the images at the end portions are the body of a viewer, the display apparatus can be utilized 
cut away may be displayed with the cut away images 25 also for such media as a DSS (Digital Satellite System), a 



30 



35 



45 



blanked as at the blanking portions 9 and 10 as seen in FIGS. 
4Ato 4C or may be displayed by extracting the same as seen 
in FIGS. 5Ato 5C. 

In particular, (7) if the turning angle of the head exhibits 
a movement amount in the leftward direction, then the 
moved position Q is present leftwardly of the center position 
P, and the original image from which the image portion 10 
or 10A at the right end portion thereof is removed is 
displayed as an image. @ If the turning angle of the head 
exhibits no movement amount and the head is directed just 
to the screen, then the center position P and the moved 
position Q are the same as each other, and the original image 
from which the image portions 9 and 10 or 9A and 10A at 
the left and right end portions thereof are removed is 
displayed as an image. (5) If the turning angle of the head 40 
exhibits a movement amount in the rightward direction, then 
the moved position Q is present rightwardly of the center 
position P, and the original image from which the image 
portion 9 or 9 A at the left end portion thereof is removed is 
displayed as an image. 

The center position P is the center position of an extracted 
image and also the center position for audio processing, and 
the moved position Q indicates a normal position of an audio 
image with respect to the center position P of the image. 
Accordingly, if the positions P and Q shown in FIG. 4B or 50 
5B are made coincide with each other, then sound which 
follows up a varied image can be obtained. 

It is to be noted that, while only those elements for a right 
audio signal are shown in FIG. 10, a convolution integrator 
32, a memory 33, a digital to analog converter 34 and a 55 
sounding body 35 are provided also for a left audio signal 
and the impulse response signal from the memory 31 is 
supplied to the convolution integrator 32 and the memory 
33. 

While the present invention is described above in con- 60 
nection with the preferred embodiments thereof, the present 
invention is not limited to the specific embodiments, and 
according to the present invention, by varying a display 
screen to be displayed from an original image in response to 
a movement of a part of the body of a viewer such as a 65 
movement of the eye-balls or a movement of the entire body 
and varying the sound in response to the thus varied screen, 



DVD (Digital Video System) and DVB (Digital Video 
Broadcasting) and for control of a game by making effective 
use of angle information used for a movement of a video 
image or an audio image. 

Having now fully described the invention, it will be 
apparent to one of ordinary skill in the art that many changes 
and modifications can be made thereto without departing 
from the spirit and scope of the invention as set forth herein. 
What is claimed is: 

1. A display apparatus operable in a synchronized rela- 
tionship with a movement of the body of a viewer, com- 
prising: 

display means for displaying a portion of an original 
image on a screen; 

angle calculation means including means for detecting 
one of a movement amount and a turning angle of a part 
of the body of a viewer of said display means; 

video processing means for extracting a portion of said 
original image in response to one of said movement 
amount and said turning angle and supplying said 
extracted portion of said original image to said display 
means; 

audio reproduction means; and 

audio processing means connected to said audio repro- 
duction means for processing an audio signal in 
response to one of said movement amount and said 
turning angle calculated by said angle calculation 
means so that sound is output from a position corre- 
sponding to said portion of said original image 
extracted by said video processing means, wherein said 
audio processing means includes: 
storage means connected to said angle calculation 

means for storing a table correlating said turning 

angle to an impulse response representing a virtual 

sound source position, and 
convolution integrator means connected to said storage 

means for convolutionally integrating said impulse 

response with said audio signal. 

2. A display apparatus according to claim 1, wherein the 
original image is obtained by reproduction of a compressed 
digital video signal, and said image processing means 
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decodes a portion of the digital video signal to extract the 
portion of the original image. 

3. A display apparatus according to claim 1, wherein said 
display means includes a head mounted display unit. 

4. A display apparatus according to claim 3 7 wherein said 
detection means includes a gyro sensor. 

5. A display apparatus according to claim 1, wherein said 
display means includes a head mounted display unit, and 
said audio reproduction means includes a headphone. 

6. A display apparatus according to claim 5, wherein said 
detection means includes a gyro sensor. 



14 



7. A display apparatus according to claim 1, wherein said 
display means includes a projector. 

8. A display apparatus according to claim 1, wherein said 
display means blanks any other portion of the original image 
than the portion to be displayed in order to extract the 
portion to be displayed. 

9. A display apparatus according to claim 1, wherein said 
detection means detects a movement amount or a turning 
angle of the head of the viewer. 
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